Transform-Domain Solutions of Poisson's Equation with Applications to the Asymptotic Variance

Recent years have seen a considerable increase of attention devoted to Poisson’s equation for Markov chains, which now has attained a central place in Markov chain theory, due to the extensive list of areas where Poisson’s equation pops up: perturbation analysis, Markov decision processes, limit theorems of Markov chains, etc. all find natural expression when viewed from the vantage point of Poisson’s equation. We describe how the use of generating functions helps solve Poisson’s equation for different types of structured Markov chains and for driving functions, and point out some applications. In particular, we solve Poisson’s equation in the transform domain for skip-free Markov chains and Markov chains with linear displacement. Closed-form solutions are obtained for a class of driving functions encompassing polynomial functions and functions with finite support.